Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cuda api tracer tool to accel-sim framework #162

Open
wants to merge 6 commits into
base: dev
Choose a base branch
from

Conversation

William-An
Copy link

Add CUDA runtime API tracer. Able to track:

  1. CUDA memory allocation call
    1. Recore pointer allocation sizes
    2. Store device pointers as symbolic names and reference in later calls
  2. CUDA memcpy call
    1. Identify memcpy direction
    2. Identify host and device (symbolic name) pointers
    3. Identify copy size
    4. Dump memcpy data to file
  3. CUDA kernel launch call
    1. Identify kernel name and mangled ptx name
    2. Identify kernel function pointer
    3. Identify grid and block size
    4. Identify shared bytes and stream
    5. Identify arguments and argument sizes
      1. If a device pointer, log the symbolic name
      2. If a constant, log the value
      3. Currently use a naive approach to identify arguments with only basic data types (int, float, double, and their 1D pointers)
        1. Waiting on the next release of NVBit to resolve a bug on retrieving argument size info
        2. NVBit issue: nvbit_get_kernel_argument_sizes get error CUDA_ERROR_INVALID_VALUE NVlabs/NVBit#80
  4. CUDA memory free call

@JRPan
Copy link
Collaborator

JRPan commented Feb 10, 2023

One thing I'm going to complain about is identifying data types. Is this the only way? It will instantly fail if an application uses some custom data type.

@William-An
Copy link
Author

Yeah that is fair. I guess there are two ways to do this:

  1. Wait on the NVBit tool for them to fix the issue so we get the sizes of each argument.
  2. Or we could ask the user to provide some sort of mapping between data types that go into the kernels and the corresponding sizes.

But ultimately still have to perform some checks on whether the argument is a pointer or not, which could only be confirmed by checking the function signature?

@William-An William-An requested a review from tgrogers February 11, 2023 03:14
@JRPan
Copy link
Collaborator

JRPan commented Feb 11, 2023

Can you add an else if block at the end and just asssert(0) in it? So if the data type is not enumerated then it will just break and the user would know where to look at. And add a comment there telling the user to add their own data types if reaching the asssert(0).

@William-An
Copy link
Author

William-An commented May 6, 2024

For the data type identification, we can use the PTX parser in gpgpu-sim to parse it, though it is somewhat hacky.
Update: or we can do this: NVlabs/NVBit#80 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants